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Abstract 


The purpose of this study was to investigate the impact of two innovative, art-and-literature-based 
approaches to teaching writing (Picturing Writing: Fostering Literacy Through Art and Image-Making 
Within The Writing Process) on elementary school students’ writing, visual literacy, and reading skills. 
RMC Research Corporation of Portsmouth, New Hampshire designed and conducted a three-year quasi- 
experimental study with matched comparison groups for the Manchester School District (MANSD) in 
Manchester, NH to determine the effect of an enhanced integrated PW/IM model on students’ 
academic performance. Three elementary schools served as treatment schools with three 
demographically matched elementary schools in the district serving as comparison schools. The study 
sample consisted of about 1500 students each year, grades 1-4, and included two ELL Magnet strands, 
grades 1-5. Because the City of Manchester is a national refugee resettlement community, MANSD 
serves families with the highest poverty and the most diverse student population in the state. MANSD 
was identified as a "District In Need of Improvement" under the No Child Left Behind Act just prior to 
commencing the study. Pre- and post-test art-and-writing samples were collected from all participating 
students in the fall and spring of each year. All identifying information was removed from the samples. 
Spring samples consisted of narratives that included one or more pictures. Student writing was 
separated from the art and typed in a uniform fashion, insuring a blind study. Two separate scoring 
instruments, developed for previous studies, were refined to meet the needs of a wider range of grade 
levels. To measure additional student academic outcomes, the study used State and District measures 
including the NH statewide New England Common Assessment Program (NECAP), the Gates-MacGinitie 
Reading Comprehension Tests, and district writing prompts. Within the independent evaluation, 
statistically significant gains were documented each year in writing and visual literacy for treatment 
students at all grade levels as compared to the demographically matched comparison group. This held 
true for all at-risk subgroups including “Below Benchmark Readers,” Special Education Students, ELL 
Mainstream Students, and ELL Magnet Students. Furthermore, the percentage of students who scored 
proficient or above on the NECAP Grade 5 Writing Test was higher for treatment schools as compared 
with the comparison schools, the district, and the state overall. On the final NECAP Fifth Grade Writing 
Test, boys in the high fidelity school (that benefited from consistent administrative support) scored 
equally as well as the girls; Title | students and economically disadvantaged students also demonstrated 
impressive gains over their Title | and economically disadvantaged peers in the comparison school, 
across the district, and across the state. District measures in writing also showed positive trends. Results 
from the Gates-MacGinitie did not produce clear patterns in findings. However, strong gains in reading 
on the NECAP Reading Assessments were apparent for the high fidelity treatment school. Overall 
findings suggest that a wide range of learners benefit from participating in the PW/IM models, 
particularly in the areas of writing and visual literacy. This remained true for traditionally 
underperforming groups such as “below benchmark readers,” Title | and Special Education students, 
English learners, and boys, who tend to lag behind girls in writing across the nation. Based on these 
findings, educators and policymakers should reconsider the use of traditional straight verbal teaching 
practices when it comes to the teaching of writing and visual literacy, especially given the growing 
diversity within today’s classroom. Study results are detailed within 15 exhibits and 9 tables. The two 
scoring instruments used in the independent study can be found in the Appendix. 


Introduction 


The Manchester School District (MANSD) in Manchester, New Hampshire received federal funding 
(2006-2010) through a U.S. Department of Education Arts in Education Model Development and 
Dissemination Grant to investigate the impact of Picturing Writing: Fostering Literacy Through Art (PW) 
and Image-Making Within The Writing Process (IM) on students’ writing, visual literacy, and reading 
skills. MANSD partnered with the Center for the Advancement of Art-Based Literacy at the University of 
New Hampshire to design and implement a standards-based art and language arts schoolwide spiraling 
instructional plan that was integrated into the content areas. Grade-level teaching teams selected 
science and social studies topics they wished to address using PW/IM methods while aligning units with 
grade-level standards. Schoolwide adoption of PW/IM was rolled out through enhanced professional 
development which consisted of hands-on teacher-training and classroom coaching. 


RMC Research Corporation was hired as an independent evaluator to design and conduct a research 
study that would determine through scientific evaluation the effect of the enhanced integrated model 
on students' academic performance. This report summarizes the evaluation and the effects of PW/IM on 
students’ writing achievement, students’ ability to use the visual arts as a language for communicating 
their ideas, and students’ reading achievement. 


Background 


Picturing Writing: Fostering Literacy Through Art and Image-Making Within The Writing Process are two 
dynamic art-and-literature-based approaches to literacy learning developed by Beth Olshansky at the 
University of New Hampshire. These innovative, multimodal instructional models have demonstrated 
their effectiveness in significantly improving student academic performance in writing as well as reading, 
particularly among at-risk learners, through the integration of standards-based art education into the 
core curricula. 


PW and IM have established histories of proven results in improving student achievement through 
comprehensive, systematic integration of standards-based art education and standards-based language 
arts instruction throughout the content areas (Frankel, 2011; O’Connor, 2010; Olshansky, 2007, 2008). 


In 1993, based on research findings submitted to the U.S. Department of Education’s Program 
Effectiveness Panel of the National Diffusion Network (NDN), Image-Making won validation as an 
“innovative and effective literacy program” and was awarded federal funding for national dissemination. 


A second independent research study, involving 555 first- and second-grade students from NH, HI and 
TX, was conducted from 1997-1998. This study was designed to document the impact of consistent use 
of PW/IM as a yearlong intervention. Findings demonstrated significant gains in writing achievement 
and visual literacy of participating students as compared to students in demographically matched 
comparison groups. Particularly dramatic were the documented gains in writing skills made by at-risk 
students (Title | and Special Education); notably at-risk students kept close pace with their regular 
education peers. 


In 1999-2000, the impact of PW on reading skills first became apparent. In a third-grade classroom in 
Fowler, CA, standardized test results from the John C. Fremont School revealed that 36% of students 
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scored one to four years below grade level in reading comprehension before participating in PW 
(Siverolli Reading Assessment). After eight months of consistent use of PW (2-3 times per week, 60-75 
minutes per session), 97% scored at or above grade level with 75% scoring three to four years above 
grade level. Seventy-five percent (75%) of these students received Free and Reduced Lunch (F&R); 50% 
were English Language Learners (ELLs). 


A three-year Comprehensive School Reform Demonstration (CSRD) Grant (1999-2002) for the Main 
Street School (grades K-2) in Exeter, New Hampshire to integrate PW/IM into language arts, science, and 
art curriculum provided schoolwide standardized test score data over a ten-year period (beginning the 
year prior to the grant and then tracking scores during and following the grant). Results from California 
Achievement Tests, the New Hampshire state language arts assessments (NHEIAP and later NECAP), and 
the Gates-MacGinitie Reading Comprehension Tests documented that consistent use of PW and IM asa 
schoolwide instructional model (implemented 2-3 times per week, 60-75 minutes per session) had a 
significant impact on students’ acquisition of reading and writing skills, particularly for at-risk learners. 
For example, following two years of schoolwide implementation, Title | second graders achieved and 
maintained overall increases in the percentage of students scoring in the “above average” or “high” 
range on the California Achievement Tests Total Language Arts Scores; on the NH state language arts 
assessment, Title | students consistently scored above the state average of all students in writing; and 
on the Gates-MacGinitie Reading Comprehension Test, Title | and Special Education students made 
noteworthy gains in reading, scoring above the national average of all students in reading after only two 
years of schoolwide implementation. State standardized test data over the next decade (NHEIAP and 
later NECAP) continued to document a strong performance by Exeter’s at-risk students in reading and 
writing as compared to the performance of their at-risk peers across the state. Additionally, defying 
national trends, during the five-year period when disaggregated results on gender differences were 
made available, Exeter boys outscored girls across the state in reading for four out of five years 
(O’Connor, 2010; Olshansky, 2007). 


These cumulative findings demonstrated PW/IM’s ability to level the playing field by allowing all 
students to enter the reading/writing process from a position of strength and enthusiasm. This has 
particularly profound implications for traditionally low-performing subgroups such as Title | students, 
Special Education students, boys, and English Language Learners for whom pictures serve as universal 
language. 


Picturing Writing/Image-Making Integrated Curriculum Model 

Picturing Writing utilizes a variety of simple crayon resist-based painting techniques within a progression 
of carefully crafted art-and-literature-based mini-lessons designed to teach writing and visual literacy 
skills through standards-based art and language arts instruction. Central to the process, students draft 
their ideas in pictures first using quality art materials and with the benefit of targeted art instruction. By 
analyzing the work of professional artists and writers, learning the grammar, principles, and key 
elements of each discipline through the study of quality picture books, and being given access to quality 
art materials and instruction, students learn how to create pictures that tell a story and write words that 
paint pictures. An array of simple texturing techniques (e.g., watercolor and salt, scratching into wet 
watercolor, splatter painting, sponge painting, plastic wrap prints, and tissue dabbing) in addition to the 
crayon resist process allows students to think in greater detail and therefore to access more descriptive 
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language when writing to their pictures. Students carefully craft their own quality picture books as they 
apply the art and writing skills they have learned. Their content-based picture books, designed around 
the science and social studies curriculum at each grade level, remain in the classroom library for the 
remainder of the school year, becoming favorite reading materials. Students practice reading their own 
books with fluency so that they can read them aloud to classmates and book buddies. Students are also 
eager to read each other’s books, thus creating a culture of avid readers within the classroom. 


Image-Making differs from Picturing Writing in that it uses collage made from hand-painted papers 
created by each student to provide diverse learners with concrete, visual and kinesthetic tools for 
constructing story. As students literally give shape to their ideas through moving cut and torn shapes 
across each page, they are able to rehearse, draft, and revise their stories long before setting pencil to 
paper. As they create a sequence of collage images to tell their story, students are encouraged to orally 
rehearse their storyline. As they read their pictures, students are able to access descriptive language to 
describes what is happening in each picture. 


These dynamic models represent a major paradigm shift in language arts instruction. They recognize 
that traditional straight verbal instructional methods used in classrooms across the nation are failing to 
meet the needs of our diverse learners and that many students who struggle with reading and writing 
have strengths as visual learners. PW/IM offer students a variety of visual, tactual, and kinesthetic tools 
for developing, expressing, and recording their ideas, thus eliminating the hidden bias within our schools 
that favors the verbal learner. 


Facilitated within “Artists/Writers Workshop,” the PW/IM models treat words and pictures as parallel, 
complementary, and equal languages for learning. They provide diverse learners with concrete visual 
tools for developing and expressing their ideas throughout the writing process. Within Artists/Writers 
Workshop, daily lessons are facilitated in two complementary strands: an art strand, which focuses on 
the language of pictures, and a writing strand, which focuses on the language of words. The art strand 
always precedes the writing strand, thus providing students with visual thinking tools prior to writing. 
Whether focusing on pictures or words, each lesson follows the same four-step format. 


Artists/Writers Workshop 

e Literature Share/Discussion 
e Modeling 

e Work Session 

e Group Share 


Through the use of quality picture books as mentor texts and the interweaving of literature-based art 
and writing lessons, PW and IM have demonstrated their ability to reach a wide range of learners. While 
aligned with English language arts and visual art standards, these art-infused models can be easily 
integrated across the curriculum. 


As students engage in creating their own high quality, content-based picture books, PW and IM provide 
a highly structured, enticing, alternative pathway into the reading/writing process that has proven to be 
an effective intervention, particularly for those who struggle with words. These comprehensive models 

are also aligned with many best practices including brain-based learning, differentiated instruction, 


project-based learning, self-efficacy theory, multimodal instruction, theory of multiple intelligences, and 
the zone of proximal development. Twenty-First Century Skills (critical thinking, creativity, collaboration, 
and communication) are naturally woven throughout Artists/Writers Workshop. PW/IM also offer 
students the opportunity to experience the phenomenon of transmediation (the act of recasting or 
translating of meaning from one sign system to another) known to deepen students’ thinking, generate 
new ideas, and create greater opportunities for reflection (Siegel, 1995). As students make meaning in 
pictures first and then translate that meaning into words, they are not only drawn more deeply into 
their picture, but also experience the image “as if they were there.” As they access sensory description 
to describe what is happening in their picture, students experience their picture “come alive.” 


Evaluation Design 


The AEMDD-funded evaluation was designed to test the impact of an enhanced PW/IM model on 
students’ writing, visual literacy, and reading skills within a more diverse, high-poverty educational 
setting than previously studied. While PW and IM have been widely adopted and studied in grades 1 and 
2, this most recent evaluation targeted a wider range of grade levels (grades 1-4 and two ELL Magnet 
Programs, grades 1-5) and included a much more diverse, urban population of students. Because the 
City of Manchester serves as a national refugee resettlement community, during the time of the study 
over 70 languages were spoken within MANSD. Serving the highest poverty and the most diverse 
population in NH, the entire district was identified as "In Need of Improvement" under NCLB just prior to 
receiving funding for this initiative. This study of approximately 1500 students each year over three 
years was also the largest Picturing Writing/Image-Making study of its kind to date. 


Study Design and Sample 

A quasi-experimental design with a matched comparison group was used to assess the effectiveness of 
the enhanced PW/IM model on student reading, writing and use of art as a language for communicating 
ideas in grades 1-4. Three elementary schools in MANSD served as treatment schools and three 
demographically matched elementary schools in the same district served as comparison schools. Schools 
were matched based on percentage of students receiving F&R and pre-intervention NECAP test results 
by school and grade. 


2006-2007 Data on Percentage of Students Participating in Free & Reduced Price Lunch 


% F&R % F&R 
Treatment Schools Comparison Schools 
Hallsville Elementary 44 Northwest Elementary | 47 
Webster Elementary 30 Highland Goffs Falls 26 
Smyth Road Elementary 17 Weston Elementary 23 


To insure treatment and comparison groups were appropriately matched, baseline 2006 NECAP scores 
were collected to determine if there was a difference in reading and writing scores between treatment 
and comparison schools. A review of the scores at the school level showed significant differences 
between treatment and comparison schools in reading only at Grade 3 Level 4 and Grade 4 Level 2. In 
addition, there was no difference between treatment and comparison schools in percentage of grade 5 
students writing at or above proficient levels. These results demonstrate that, for the most part, the 


8 


schools selected for the treatment and comparison groups were comparable in reading and writing 


levels. 


NECAP Tests - Percentage of Students at Different Proficiency Levels by Grade — Fall 2006 


Grade 3, 4, and 5 students in 2006-2007 


Grade 3 - Reading 


Proficiency Level 


Treatment Schools 


Comparison Schools 


Level 4** 15% 9% 

Level 3 51% 47% 

Level 2 17% 23% 

Level 1 17% 21% 
**<.05 


Grade 4 - Reading 


Proficiency Level Treatment Schools Comparison Schools 
Level 4 12% 7% 
Level 3 55% 52% 
Level 2* 18% 24% 
Level 1 15% 17% 
*p<.10 


Grade 5 - Writing 


Proficiency Level Treatment Schools Comparison Schools 
Level 4 10% 6% 
Level 3 33% 32% 
Level 2 29% 36% 
Level 1 26% 27% 


Two self-contained ELL Magnet programs within MANSD (grades 1-5) also served as treatment and 
comparison groups for evaluating the impact of the enhanced model on the academic achievement of 
this unique population of immigrant and refugee students new to the country. 


A large part of the enhanced model involved using proven PW/IM practices to design, field-test, and 
refine a schoolwide spiraling PW/IM curriculum across participating grade levels that was aligned with 
NH’s language arts and art standards and addressed MANSD’s science and social studies curriculum. The 
developer, in conjunction with a team of classroom coaches and grade-level teacher leaders, developed 
and refined a spiraling PW/IM curriculum that was implemented throughout the school year as well as 
across grade levels at the treatment schools. Students who entered the study during the primary grades 
were able to participate in the enhanced model for several consecutive years. Depending on the grade 
level and the curriculum topic, the enhanced model involved the refinement of existing units as well as 
the development of new instructional units. 


During the first year of the four-year study, teachers in the treatment schools received initial hands-on 
training and followed by classroom coaching while they piloted the model. These teachers were 
expected to implement Picturing Writing following the Artists/Writers Workshop format (described 
earlier) 2-3 times a week, each workshop session lasting for 60-75 minutes. Teachers in the comparison 
group implemented whatever writing program was being used in their school at the time. Beginning in 
Year 2 of the study, ongoing professional development occurred through an embedded coaching model. 
An annual external evaluation was conducted to examine the effectiveness of the PW/IM model. 


The study sample consisted of about 350-400 students per grade in grades 1, 2, 3, and 4 and ELL Magnet 
grades 1-5 (a total of approximately 1500 students). On average, 77 teachers participated each year 
(ranging from 68-83) from seven elementary schools during the three years of data collection. The study 
in participating schools included all students in each classroom. An ELL Magnet Program in one 
treatment school was matched with an ELL Magnet Program in a seventh school that did not already 
serve as a schoolwide comparison school. 


Data Collection Methods and Procedures 

To measure student art and writing outcomes, student pre-test and post-test art and writing samples 
were collected. Two separate scoring instruments that were originally approved by the Program 
Effectiveness Panel of the National Diffusion Network were revised, expanded, and tested to meet the 
needs of this enhanced PW/IM model for grades 1-41. One instrument focused on narrative writing skills 
(establishing a sense of setting, character development, plot development, use of descriptive language, 
and overall quality of writing, including voice and ability of words to paint pictures in the reader’s mind). 
The second instrument was used to assess the use and quality of visual elements to communicate ideas 
(use of color, texture, shape, detail, composition, sequence, and overall quality of artwork to tell the 
story). Under the advisement of a panel of veteran elementary school art and writing educators, the 
scoring instruments were reviewed, revised, and piloted during the first year of the study, and 


1 The PW/IM art and writing scoring instruments were originally developed in 1991 by Dr. Susan Frankel in 
consultation with Beth Olshansky and a team of veteran art and writing educators during a study which resulted in 
the validation of Image-Making Within The Writing Process as an innovative and effective literacy program by the 
U. S. Department of Education Program Effectiveness Panel of the National Diffusion Network. The scoring 
instruments have been refined over multiple studies, including this most recent one. 

10 


continued to be used in Years 2, 3, and 4 of the study. Special consideration was given to the NH K-12 
Curriculum Framework for the Arts, the NH State Department of Education Grade Level Expectations in 
Language Arts (GLEs) and the NH State Curriculum Frameworks for Language Arts. See Appendix A for a 
copy of the scoring instruments. Independent trained scorers used these instruments to rate the quality 
of student art and writing. 


Art and writing samples were collected in the fall (pre-test) and spring (post-test) of each year. In 
September, teachers in the treatment and comparison groups handed students a sheet of paper with a 
blank box at the top for drawing a picture and lines below for writing a story about their picture. 
Students were given 30 minutes to complete their picture and story. In May, teachers in the treatment 
and comparison schools collected stories written by students during the spring term. The post-test 
stories from the treatment students were written during the implementation of Picturing Writing or 
Image Making. The stories from the comparison group were written as part of whatever writing program 
was being used in those schools at that time. 


After each data collection period, all identifying information was removed from the student samples. 
Each story was rated on two major areas: 1) text portion of the story, and 2) use of visual elements to 
communicate ideas. An additional blind-rating structure was used to score the writing portion of the 
spring samples since students’ accompanying artwork would have identified students as belonging to 
either the treatment or comparison group. Student writing was separated from the art and typed ina 
uniform fashion, thus removing all identifying features. Independent raters with backgrounds in 
elementary education were trained to rate the text-only (written) portion of the story as well as the 
picture/word relationship (i.e. whether and how students’ pictures conveyed what was written in the 
text). Pre-test and post-test samples were rated on a scale of 1 - 6 where “1” means text or visual quality 
is not present and “6” means a text or visual quality is extraordinary. Raters received 51 hours of 
training by the Project Director and Evaluator before achieving an inter-rater reliability of 92.5%. Each 
art/writing portion and text-only portion of the story were scored by two trained raters. The average of 
the two scores was used as the student’s scores in the analyses. 


To measure additional student academic outcomes, the study used state and district reading and 
language arts assessments, including student reading and writing data on the statewide New England 
Common Assessment Program (NECAP), student reading scores on the district-wide Gates-MacGinitie 
Reading Comprehension Tests, and student writing scores on district writing prompts. Because 
classroom teachers in the district administered these tests, teachers participating in the evaluation used 
the same assessments as the other teachers in the district. 


Statewide NECAP Test writing scores. Grade 5 NECAP Writing Assessment, administered in October of 
every year, was used to further assess writing achievement. (Grade 5 was the only elementary grade 
evaluated for writing on the state assessments.) Fifth-graders in the treatment schools were exposed to 
Picturing Writing in grade 4 during the previous school year. Student NECAP writing scores, aggregated 
at the school level, were compared between treatment and comparison schools. Each of the three 
treatment schools was compared with its matched comparison school based on percentage of students 
on F&R. Findings were summarized at the school level for students at the beginning of grade 5 in the fall 
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of 2007, 2008, and 2010; Writing scores from tests taken in the fall of 2009 were not available as it was 
a “norming year.” 


District writing scores. MANSD’s writing prompt assessment was used as yet another tool to measure 
students’ writing improvement. The assessment was administered three times a year. For this study, 
assessment scores from September (time 1) and May/June (time 3) were used as pre-test and post-test 
measures of writing proficiency. The assessment measures six areas: content and ideas, organization, 
conventions, word choice, voice, and presentation. Student writing was scored on a four-point rating 
scale with “1” meaning not proficient and “4” meaning highly proficient. 


Statewide NECAP Test reading scores. The NECAP assessment contains a reading component also 
administered in October of each year. Student NECAP reading scores, aggregated at the school level, 
were compared between treatment and comparison schools. As with the NECAP writing scores, each of 
the three treatment schools was compared with its matched comparison school based on percentage of 
students on F&R. 


District reading scores. The Gates-MacGinitie Reading Comprehension Tests were also used to measure 
reading achievement in participating students. The tests were administered in June of each year and 
data were available from students in both the treatment group and the comparison group. Scores were 
reported on a four-point scale; values of 3 and 4 were combined and considered as a measure of 
proficient level of reading. 


Data Analysis and Findings 


Writing and Visual Literacy 

Art and Writing. Pre-test and post-test scores from student art and writing samples were analyzed using 
analysis of variance (ANOVA) to test differences between treatment and comparison groups. Separate 
analyses were conducted on scores for each quality as measured by a scale or item from the scoring 
instruments, and for each grade and across all grades for treatment effect. 


In the first year of the evaluation, statistically significant gains in writing were achieved by students in 
the treatment group as compared to the demographically matched comparison group across all grade 
levels. Both the comparison group and treatment group began in September with similar scores of 1.31 
and 1.43, respectively, for the overall quality of student writing (scores that reflect less than minimal 
levels of achievement in basic writing skills), but by the spring dramatic differences in writing skills were 
apparent between the two groups. Although there was growth within the comparison group, much 
greater gains were made by students participating in the treatment. The average score for the overall 
quality of student writing of the treatment group jumped from 1.43 to 4.80 (nearly “fully developed”). 
The average score of the comparison group increased slightly to 1.85, remaining a minimal score. See 
Exhibit 1. Findings were similar for each of the four grade levels. 
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Exhibit 1. Pre-Post Test Writing Scores, All Grade Levels, 2007-2008 


m Comparison (n=673) Treatment (n=692) 
6.00 - 


5.00 - 4.80 


4.00 + 


3.00 + 


PW Writing Score 


2.00 + 


1.00 + 


0.00 - 


Pre-test (Fall 07)* Post-test (Spring 08)* 


*The difference between comparison and treatment groups is statistically significant. 


The findings in visual literacy showed a similar pattern in students’ ability to communicate their ideas 
through the use of visual elements; statistically significant gains were made by students participating in 
the treatment while only slight gains were made by students in the demographically matched 
comparison group. Students from both groups began the school year with roughly the same level of skill 
(minimal scores of 1.60 for the comparison group and 1.74 for the treatment group) for overall use and 
quality of visual information. After participation in the treatment, the overall average score of the 
treatment students increased to 4.42 (display of trait is partially developed). The average score of 
students in the comparison group remained at the low end of the continuum with 1.80 (Exhibit 2). 
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Exhibit 2. Pre- and Post-Test Visual Literacy Scores, All Grade levels, 2007-2008 


m Comparison (n=674) m@ Treatment (n=688) 
6.00 - 


5.00 + 


4.42 


4.00 - 


3.00 - 


PW Visual Score 


2.00 - 


1.00 + 


0.00 + 
Pre-test (Fall 07)* Post-test (Spring 08)* 


*The difference between comparison and treatment groups is statitically significant. 


This pattern also remained consistent for all subgroups of students targeted by No Child Left Behind: At- 
Risk Students (defined as students who scored below benchmark in reading in the fall of that year), 
Special Education Students, ELL Mainstream Students, and ELL Magnet Students. All subgroups made 
statistically significant gains in Year One as compared to their demographically matched comparison 
groups, and kept close pace to “all students” in the treatment group. (“All students” included regular 
education students plus various at-risk subgroups.) Note that even ELL Magnet Students (relatively new 
to the country) scored far greater than “all students” in the comparison group (Exhibit 3). 
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Exhibit 3. A Comparative Summary of PW Spring Writing Scores for Subgroups Across Grade Levels, 
Spring 2008 


m Comparison @ Treatment 


4.80 
5.0 - 4.61 4.60 ie 


PW Writing Scores 


All Students At-Risk Students SPED ELL Mainstream ELL Magnet 
(nc=673; nt=692)*(nc=203; nt=183)* (nc=87; nt=63)* (nc=53; nt=43)*  (nc=42; nt=24)* 


*The difference between comparison and treatment groups is statistically significant. 


Findings followed a similar pattern when looking at the Comparative Summary of Visual Scores for All 
Subgroups: Spring 2008 (Exhibit 4). 
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Exhibit 4. A Comparative Summary of PW Spring Visual Literacy Scores for Subgroups Across Grade 
Levels, Spring 2008 


m=Comparison sm Treatment 


6.0 


j=) 


RW Visual Scores 
oO 


j=) 


0.0 


All Students At-Risk Students SPED ELL Mainstream ELL Magnet 
(nc=674; nt=688)* (nc=206; nt=181)* — (nc=86; nt=61)* (nc=53; nt=43)* 
*The difference between comparison and treatment groups is statistically significant. 


This pattern of findings remained consistent over the course of the three years of the evaluation. Using 
the independently developed scoring instruments to evaluate students’ story writing and use of visual 
elements to communicate ideas, significantly greater gains in writing and visual literacy were achieved 
by treatment students at all grade levels each year as compared to the demographically matched 


comparison group. 


Exhibit 5 shows the spring writing scores each year for subgroups of students in the treatment group. 
Each year, after consistent participation in the treatment, students in various subgroups increased their 
overall writing scores, with minor variations seen among SPED and ELL Magnet populations dependent 
on the severity of these particular students’ learning needs, or in the case of the Magnet ELL students, 
whether or not they had any previous formal education. Within the treatment group, the overall spring 
writing scores for “all students” ranged between 4.82 and 4.94. Note that all subgroups, except for 
those students who were brand new to the country, kept close pace with “all students” by the spring of 
each year with the ELL Mainstream Students outscoring “all students” in writing in 2009 and 2010. 
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Exhibit 5. PW Spring Writing Scores, Treatment Students - Subgroups: 2008, 2009, 2010 


6.0 


5.0 4.82 4.88 4.94 4.84 4.95 4.97 


3.82 3.94 


4.0 

; 3.30 
3.0 [ 
2. [ 
1.0 


0.0 


PW Writing Score 


[o) 


2008 2009 2010 2008 2009 2010 2008 2009 2010 2008 2009 2010 2008 2009 2010 
(n=668) (n=663) (n=682) (n=183) (n=134) (n=168) (n=62) (n=58) (n=78) (n=43) (n=41) (n=52) (n=24) (n=30) (n=25) 
All Students At-Risk Students SPED ELL Mainstream ELL Magnet 


Spring scores on the use of visual elements for at-risk students in the treatment group across the three 
years of the study were consistently high compared to comparison groups though certainly displayed 
more variation year by year. As noted in our conclusion section, in school year beginning in September of 
2009 there was a radical turnover in teaching staff, thus many teachers during this year (except for the 
ELL Magnet teachers) would have been new to facilitating the treatment (Exhibit 6). 


Exhibit 6. PW Spring Visual Literacy Scores, Treatment Students - Subgroups: 2008, 2009, 2010 
6.0 


5.0 


= 
ro) 


WW 
oO 


N 
oO 


PW Visual Literacy Score 


1.0 
0.0 
2008 2009 2010 2008 2009 2010 2008 2009 2010 2008 2009 2010 2008 2009 2010 


(n=664) (n=663) (n=682) (n=181) (n=133) (n=168) —(n=60) (n=57) (n=78) (n=43) (n=42) (n=52) (n=24) (n=30) (n=25) 
All Students At-Risk Students SPED ELL Mainstream ELL Magnet 
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Analysis by gender was conducted in the final year. Differences in writing between boys and girls were 
minimal. The average spring scores for the overall quality of writing for both boys and girls participating 
in the treatment groups were far greater than the scores for boys and girls in the comparison groups. 
The spring scores for boys and girls in the comparison group were 2.02 and 2.15, respectively. The spring 
scores for boys and girls in the treatment group were 4.89 and 4.97, respectively. These findings suggest 
that boys, who typically underperform in the area of language arts across the nation, are more 
successful with hands-on visual and kinesthetic language arts instruction, keeping close pace with their 


female counterparts (Exhibit 7). 


Exhibit 7. PW Spring Writing Scores Treatment vs. Comparison Students, Girls and Boys: 
All Grade Levels 2009-2010 


6.00 mGirls m Boys 


4.97 agg 


5.00 


4.00 


3.00 


PW Writing Score 


2.00 


1.00 


0.00 
Comparison Treatment 
(n=654) (n=682) 
*The difference between comparison and treatment groups is statistically significant, p<O00. 
**The difference between females and males is statistically significant, p<.05. 


Writing 

Writing Scores on State Assessment (NECAP). Table 1 shows results of Grade 5 NECAP Writing Test. 
Findings are summarized at the school level for students at the beginning of grade 5 in the fall of 2007, 
2008, and 2010; students in the treatment schools participated in the enhanced PW/IM model in grade 
4 during the prior year, although fifth graders taking the NECAPs in the fall of 2007 only participated in 
PW for only 5 months beginning in January of that year. Writing scores were not available for the fall of 
2009 because 2009-2010 was a statewide “norming” year for the writing assessment. 
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Table 1. Comparison of Students Scoring at Proficiency or Above on 2007, 2008, and 2010 NECAP 
Grade 5 Writing Test Between Comparison and Treatment Schools 


Fall 2007 


Fall 2008 


Fall 2010 


Comparison 
Schools 


Picturing 
Writing 
Schools 


District 


Comparison 
Schools 


Picturing 
Writing 
Schools 


District 


Comparison 
Schools 


Picturing 
Writing 
Schools 


District 


Number of 
students in 
grade 5 at 
beginning of 
Fall who took 
NECAP writing 
test 


320 192 1,100 286 202 1,106 279 151 1,020 


Number of 
students who 
scored at 
proficient or 
above 
proficient 
levels 


101 Tl 343 96 104 419 93 93 424 


Percentage of 
students who 
scored at 
proficient or 
above 
proficient 
levels 


32% 37% 31% 33% 51% 38% 36% 61% 42% 


The percentage of students proficient on the Grade 5 NECAP Writing Test was higher for schools that 
implemented the treatment than for the comparison schools, and the district overall, with the gap 
widening over time. In the fall of 2008 on the NECAP Writing Assessment, 51% of students in the 
treatment group scored proficient compared to 33% of students in the demographically matched 
comparison group whereas in the fall of 2007 only 37% of students in the treatment group scored 
proficient compared to 32% of students in the demographically match comparison group. In 2010, the 
final year of our study, 61% of treatment students scored proficient while 36% of comparison students 
scored proficient. 


When looking at the standardized test results in writing for the treatment schools as compared to 
demographically matched comparison schools, some interesting findings emerge. While one might 
assume that the school with the strongest test results was the treatment school with the lowest 
percentage of students on F&R, in this case the T1/C1 partner schools had the highest percentage of 
students on F&R (54% and 52% respectively). Closer analysis revealed that this treatment school was the 
only treatment school with consistent administrative support throughout the four-year research period 
(Exhibit 8). The other two schools experienced turnover in administration, resulting in a lack of strong 
leadership regarding implementation of the treatment. These two schools also experienced greater 
turnover in teaching staff. 
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Exhibit 8. Percentage of Fifth Graders Scoring Proficient or Above on NECAP Writing 2010 by 
Treatment/Comparison Schools 


mTreatment m Comparison 


80% 


70% 


60% 


50% 


40% 


30% 


20% 


10% 


0% 
T1/C1 T2/C2 T3/C3 


Defining T1 as the “high fidelity school” based on its consistent administrative support, it is informative 
to drill down deeper to look at the potential of PW/IM when its art-based writing process is fully 
supported by administrators and thus implemented with fidelity. 


Exhibit 9 shows the writing gains over time of the high fidelity treatment school as compared to its 
demographically matched comparison school on the NECAP State Writing Assessment. 


Exhibit 9. Percentage of Students Proficient and Above on Grade 5 NECAP Writing Test 
@ Northwest m Hallsville 

80% 

70% 

60% 

50% 


40% 


Percent of Students 


30% 


20% 


10% 


0% 


2007-2008 2008-2009 2010-2011 
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Most interesting are the findings for boys versus girls on the NECAP Writing Assessment in the final year 
of the study. Note that boys and girls scored identically and identically well in writing. This did not 
happen in the comparison school, anywhere in the District, nor anywhere in the State (Exhibit 10). This 
is a particularly interesting finding given that boys typically lag behind girls in language arts across the 
nation (National Center for Education Statistics, 2000). 


Exhibit 10. Grade 5 NECAP Writing Test 2010-2011 Percentage Proficient and Above by Gender 


HGirls Boys 
80% 


73% 73% 


70% 


60% 


50% 


40% 


30% 


20% 


10% 


0% 
Hallsville Northwest District State 
(Treatment) (Comparison) 


This finding is also interesting in the context of the brain research conducted by social scientist Michael 
Gurian, who has observed physiological differences between the brains of boys and girls. Given that 
Gurian discovered that boys have more cortical area of the brain dedicated to spatial functioning while 
girls have more cortical area of the brain dedicated for verbal functioning, this finding regarding the 
performance of boys participating in a visual/spatial approach to writing is of particular interest (Gurian, 
2005). 


Also of interest are the comparative findings for Title | students (Exhibit 11) and Economically 
Disadvantaged students (Exhibit 12). 
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Exhibit 11. Grade 5 NECAP Writing Test 2010-2011 Percentage Scoring Proficient and Above: 
Title | Data 


80% 


73% 


70% 


60% 


50% 


40% 


30% 


20% 


10% 


0% 


Hallsville Northwest District State 
(Treatment) (Comparison) 
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Exhibit 12. Grade 5 NECAP Writing Test 2010-2011 Percentage Scoring Proficient and Above: 
Economically Disadvantaged Data 


90% 


80% 


70% 


60% 


50% 


40% 


30% 


20% 


10% 


0% 


Hallsville Northwest District State 
(Treatment) (Comparison) 


District Writing. The Manchester NH School District’s Writing Prompt assessment was also used to 
measure students’ writing improvement. Student writing was scored on a four-point rating scale with 
“1” being not proficient and “4” being highly proficient.* The assessment measures six areas: content 
and ideas, organization, conventions, word choice, voice, and presentation. 


For each year of the research study, pre-post matched data were available from students in the 
treatment and the comparison groups. Across the three years, on most of the writing areas, students 
participating in the treatment scored higher than their comparison peers. One consistent finding across 
the years was that students participating in the treatment scored higher than students in the 
comparison group in the writing area of voice, and the difference across all grade levels was statistically 
significant each year. Tables 2, 3 and 4 show the findings for all students by year and grade level. 


? Teachers were responsible for scoring their own students’ writing, thus the results may not rise to the same 
standard of validity as assessments that are scored by independent raters who are trained to meet high levels of 
inter-rater reliability. 
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Table 2. Table Student Performance on School District’s Writing Prompt by Treatment Group — 
All Students (Grades 1-4), 2007-2008 


Trait of Writing Comparison (n=706) Treatment (n=705) Fy Fa Effect Size 
Mean Std. Adj. Mean Std. Adj. 
Dev Mean Dev Mean 


Content 
Fall 2007 2.06 82 2.05 .80 .04 
Spring 2008 


Organization 
Fall 2007 
Spring 2008 


Conventions 
Fall 2007 
Spring 2008 

Word Choice 
Fall 2007 
Spring 2008 


Voice 
Fall 2007 
Spring 2008 
Presentation 
Fall 2007 
Spring 2008 : : 13.78** 


F; tests the difference on pretest (fall 2007) between comparison and treatment groups. 
F2 tests the difference on posttest (spring 2008) between comparison and treatment groups after adjusting pretest differences. 
*p<.01; **p<.001. 
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Table 3. Table Student Performance on School District’s Writing Prompt by Treatment Group — 
All Students (Grades 1-4), 2008-2009 


Comparison (n=717) Treatment (n=665) 
Trait of Writing 


Std. Dev ; Std. Dev 


Content 
Fall 2008 : : 15.21*** 
Spring 2009 

Organization 
Fall 2008 
Spring 2009 

Conventions 
Fall 2008 
Spring 2009 

Word Choice 
Fall 2008 
Spring 2009 

Voice 
Fall 2008 : j 42.12*** 
Spring 2009 

Presentation 
Fall 2008 : : 10.53** 
Spring 2009 

F, tests the difference on pretest (fall 2008) between comparison and treatment groups. 


Fz tests the difference on posttest (spring 2009) between comparison and treatment groups after adjusting pretest differences. 
*p<.05; **p<.001; ***p<.001. 
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Table 4. Table Student Performance on School District’s Writing Prompt by Treatment Group — 
All Students (Grades 1-4), 2009-2010 


Comparison (n=667) Treatment (n=680) 
Trait of Writing Adi Fy Fo 
Mean Std. Dev | Adj. Mean Mean Std. Dev : 
Mean 
Content 
Fall 2009 1.98 89 2.09 85 5.76* 
Spring 2010 2.81 82 2.82 2.95 79 2.93 6.83** 


Organization 
Fall 2009 
Spring 2010 

Conventions 
Fall 2009 
Spring 2010 

Word Choice 
Fall 2009 
Spring 2010 

Voice 
Fall 2009 


Spring 2010 


Presentation 
Fall 2009 
Spring 2010 


F; tests the difference on pretest (fall 2009) between comparison and treatment groups. 


F tests the difference on posttest (spring 2010) between comparison and treatment groups after adjusting pretest differences. 
*p<.05; **p<.01; ***p<.001. 
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Further analysis showed that in 2008-09 and 2009-10, at-risk students in the treatment group across all 
grade levels scored higher than their peers in the comparison group in all six writing areas. In 2007-08 
at-risk students participating in the treatment scored higher than students in the comparison group in 
five of the six writing areas. Tables 5, 6, and 7 show the findings for at-risk students by year and across 


grade levels. 


Table 5. Table Student Performance on School District’s Writing Prompt by Treatment Group — 
At-Risk Students (Grades 1-4), 2007-2008 


Comparison Treatment 


ne Std. Dev n Bo: Std. Dev 4 
Writing — Content 
Fall 2007 
Spring 2008 7.04* 
Writing — 


Organization 
Fall 2007 
Spring 2008 
Writing — 
Conventions 
Fall 2007 
Spring 2008 
Writing — Choice 
Fall 2007 
Spring 2008 


.66 


42 


Writing — Voice 
Fall 2007 
Spring 2008 

Writing — 

Presentation 
Fall 2007 
Spring 2008 

F; tests the difference on pretest (fall 2007) between comparison and treatment groups. 

F2 tests the difference on posttest (spring 2008) between comparison and treatment groups after adjusting pretest differences. 

*o<.01; **p<.001. 


7.57* 


1.45 
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Table 6. Table Student Performance on School District’s Writing Prompt by Treatment Group — 
At-Risk Students (Grades 1-4), 2008-2009 


Comparison (n=204) Treatment (n=136) 


Trait of Writing Fy Fo 


Adj. 


Mean Std. Dev | Adj. Mean Mean Std. Dev 
Mean 


Content 
Fall 2008 
Spring 2009 


Organization 
Fall 2008 
Spring 2009 


Conventions 
Fall 2008 
Spring 2009 
Word Choice 
Fall 2008 
Spring 2009 


Voice 
Fall 2008 1.39 .68 1.60 72 7.25* 
Spring 2009 2.19 77 2.21 2.38 75 2.34 2.34 


Presentation 
Fall 2008 1.57 .83 1.70 85 1.84 
Spring 2009 2.49 .86 2.52 2.45 .88 2.42 1.01 


F; tests the difference on pretest (fall 2008) between comparison and treatment groups. 


F2 tests the difference on posttest (spring 2009) between comparison and treatment groups after adjusting pretest differences. 
* 
p<.01. 
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Table 7. Table Student Performance on School District’s Writing Prompt by Treatment Group — 
At-Risk Students (Grades 1-4), 2009-2010 


Treatment (n=171) 


Comparison (n=165) 


Trait of Writing 


Adj. 
Mean 


Mean Std. Dev | Adj. Mean Mean Std. Dev 


Content 
Fall 2009 
Spring 2010 


Organization 
Fall 2009 
Spring 2010 

Conventions 

Fall 2009 

Spring 2010 

Word Choice 
Fall 2009 
Spring 2010 

Voice 

Fall 2009 

Spring 2010 


Presentation 
Fall 2009 
Spring 2010 


F; tests the difference on pretest (fall 2009) between comparison and treatment groups. 


F2 tests the difference on posttest (spring 2010) between comparison and treatment groups after adjusting pretest differences. 
*p<.05; **p<.005. 


Reading 
While Picturing Writing and Image-Making are instructional models used to teach writing, because the 
reading and writing processes are so closely linked, our study tracked state and district reading scores. 


Reading Scores on State Assessment (NECAP). Table 8 presents the results of the NECAP Reading Test 
over three years of the evaluation. Findings are summarized at the school level for students at the 
beginning of grades 3-5 in the fall of 2007, 2008 and 2009. In the fall of 2008 and 2009, students in the 
treatment schools participated in PW/IM in grades 2-4 during the prior year. In the fall of 2007, 
however, students in the treatment schools participated in PW for less than 5 months in grades 2-4 in 
the previous year. 
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Table 8. A 3-Year Comparison of Students Scoring at Proficiency or Above on NECAP Reading Tests 
Between Comparison and Treatment Groups 


Number of 
students in 
grades 3-5 at 
beginning of 
Fall who took 
NECAP reading 
tests 


Fall 2007 


Fall 2008 


Fall 2009 


Fall 2010 


Comparison 
Schools 


918 


Picturing 
Writing 


Schools 


591 


Picturing 
Writing 
Schools 


Comparison 
Schools 


872 555 


Picturing 
Writing 
Schools 


Comparison 
Schools 


847 542 


Picturing 
Writing 
Schools 


Comparison 
Schools 


866 553 


Number of 
students who 
scored at or 
above 
proficient levels 


554 


356 


526 359 


561 369 


548 358 


Percentage of 
students who 
scored at or 
above 
proficient levels 


60% 


60% 


60% 65% 


66% 68% 


63% 65% 


Findings show that in the fall of 2007, the percentage of students who were proficient on the NECAP 


Reading Test was the same for the treatment and comparison schools (60%) indicating that at the 


beginning of the research study, students in the demographically matched schools had similar 


percentages of students reading at or above proficient levels. In subsequent years, the percentage of 


students reaching proficiency was slightly higher for schools implementing the treatment than for 


comparison schools. 


When looking at the NECAP reading results for the “high fidelity” school (the school with consistent 


administrative support and more consistent teaching staff) we see a stronger pattern emerge. From 


2008-2010, the treatment school’s NECAP reading scores show consistent growth in reading by Grade 5 


whereas the comparison school’s scores show erratic results (Exhibit 13). 


30 


Exhibit 13. Percentage of Student Proficient and Above on Grade 5 NECAP Reading Test 


@ Northwest MHallsville 
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District Reading. The Gates-MacGinitie Reading Comprehension Test was used to measure reading 
achievement. This multiple-choice test was annually administered in June and scored by classroom 
teachers. Scores were reported on a four-point scale, values of 3 and 4 indicate that students were 
proficient at reading. 


Table 9 presents the percentage of proficient students in both treatment and comparison groups for 
each year of the research study and by grade level. Across all three years, the percentage of students 
scoring proficient in grade 4 was higher for the treatment group than for the comparison group. For 
grades 1-3, the findings differed by grade level and by year, with dips in treatment scores corresponding 
in part to high incidences of teacher turnover. In 2009, the difference between the percentage of grade 
1 students scoring proficient in the treatment and comparison groups was statistically significant, with 
students participating in the treatment scoring higher than their comparison peers. 
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Table 9. Student Performance on Gates-MacGinitie Reading Tests By Comparison and Treatment 
Groups and Grade — 2008-2010 


Comparison Treatment 
Mes Uren aiads % Proficient % Proficient 

Grade 1 

2008 75% 70% 

2009** 68% 83% 

2010 76% 73% 
Grade 2 

2008 71% 74% 

2009 76% 73% 

2010 72% 79% 
Grade 3 

2008 74% 78% 

2009 76% 66% 

2010 74% 66% 
Grade 4 

2008 65% 70% 

2009 70% 76% 


X? tests the difference between comparison and Picturing Writing groups. 
*p<.01; **p<.001 


Summary of Findings 


Overall outcomes from the evaluation findings were as follows: 


Within the independent evaluation, statistically significant gains were documented each year in 
writing for treatment students at all grade levels as compared to the demographically matched 
comparison group. This held true for all at-risk subgroups as well, including “Below Benchmark 

Readers,” Special Education Students, ELL Mainstream Students, and ELL Magnet Students. 


In that same study, statistically significant gains were documented each year in visual literacy for 
treatment students at all grade levels as compared to the demographically matched comparison 
group. This held true for all at-risk subgroups as well, such as “Below Benchmark Readers,” 
Special Education Students, ELL Mainstream Students, and ELL Magnet Students. 


The performance of boys in the treatment group was comparable to the performance of girls in 
the treatment group. These findings suggest that boys (who traditionally lag substantially behind 
girls in the area of writing across the nation) are more successful when participating in PW’s and 
IM’s visual, tactual, and kinesthetic approaches to writing. 


The percentage of students who scored proficient or above on the NECAP Grade 5 Writing Test 
was higher for schools that implemented PW/IM compared with the comparison schools, the 
district, and the state overall. In the fall of 2010 NECAP Writing results, which reflect the last 
year of student participation in PW/IM within the grant period, 61% of the students in the 
treatment group scored proficient or above as compared to 36% of students in the 
demographically matched comparison group. This compared to 42% of the students across the 
district scoring proficient or above and 56% across the state. 
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While differences among the three treatment school scores were apparent, the high fidelity 
school (the school with consistent administrative support and less teacher turnover) 
demonstrated the strongest results in writing, and particularly for those subgroups that typically 
perform poorly on standardized assessments. On the 2010 NECAP Fifth Grade Writing, boys in 
the high fidelity school scored equally well as the girls; Title | students and economically 
disadvantaged students also demonstrated impressive gains over their Title | and economically 
disadvantaged peers in the comparison school, across the district, and across the state. 


Exhibit 14 shows 2010 comparative writing results for fifth graders across the district, with our 
high fidelity school scoring top in the district despite the fact that 8 of the lower performing 
schools had a smaller percentage of students on F&R. 


Exhibit 14. Grade 5 NECAP Writing Test 2010-2011, Percentage Scoring Proficient and Above 
Across the District 
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e District measures in writing also showed overall positive trends. The findings differed by grade 


levels. One consistent finding across all grades over the three years was that the treatment 


students scored higher than their comparison peers in the writing area of voice; the difference 


was Statistically significant each year. 


33 


80 


e Findings from standardized reading tests (Gates-MacGinitie) varied grade level to grade level 
and test-by-test and, as a result, the reading data collected did not produce clear patterns in 
findings. Overall, the reading assessment data (Gates-MacGinitie and the state NECAP Reading 
Assessment) did not show statistically significant differences between students’ scores in the 
treatment classrooms and students’ scores in the comparison classrooms. However, strong 
gains in reading on the NECAP Reading Assessments were apparent for the high fidelity 
treatment school. This school scored second in the district in reading on the 2010 NECAP 
Reading Assessment. The top scoring school served the most privileged population in the district 
(with only 15% of students on F&R) while our high fidelity treatment school served a much more 
high-need population (54% of students on F&R). 


Exhibit 15: NECAP Reading 2010-2011 Percentage Scoring Proficient and Above: Grade 5 
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Conclusions 


Our data reveal that students in grades 1-4 benefit greatly from participating in the enhanced Picturing 
Writing/Image-Making art-and-literature-based literacy model in the areas of writing and visual literacy. 
As a result of this AEMDD-funded research study, we have found that when students are given PW/IM’s 
highly structured visual-verbal approach to literacy learning and thus two “languages” to think, develop, 
and record their ideas (the language of pictures and the language of words), they display deeper levels 
of thinking and greater power of expression in both pictures and words. Through the power of 
transmediation (in this case, translating or recasting meaning from pictures to words), it is clear that 
encouraging students to think in detailed pictures first before they write helps to deepen their thinking 
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and provides access to greater detail and description in their writing. We found that the highly 
structured visual-verbal intervention provided by PW and IM is particularly helpful to at-risk learners 
such as “Below Benchmark Readers,” Special Education students, and English Language Learners for 
whom pictures serve as a universal language. Students who, for one reason or another, struggle with 
writing have demonstrated that when they are able to utilize the visual thinking strategies offered by 
PW/IM, their verbal skills are strengthened. Given the pressure to have all students gain critical literacy 
skills and perform well on standardized tests, these data will be both useful and important. 


These positive outcomes also bore out in the state writing data collected and analyzed. On the final 
NECAP Writing Assessment (2010), 25% more fifth graders who participated in the treatment prior to 
fifth grade reached proficiency or above in writing than their comparison peers. Additionally, NECAP 
Writing Assessment data regarding traditionally underserved populations (boys, Title |, and Economically 
Disadvantaged) offer great promise for those students who have repeatedly demonstrated that they do 
not perform well when using traditional straight verbal approaches. 


With its documented positive results for at-risk populations, who tend to display more visual and less 
verbal acuity, the enhanced PW/IM model offers a viable alternative instructional model. This is 
particularly relevant, given the number of schools and districts across the nation that have been 
identified as low performing schools and the importance of writing as an essential literacy skill. 


Additionally, with the growing number of immigrant and refugee students in our schools, and given that 
pictures offer a universal language for thinking and recording ideas, documented success with English 
Language Learners provides important data in determining future policy and instructional practices. PW 
and IM give teachers critical new tools for reaching all their students, even those with limited English 
proficiency. 


There also has been a great deal of discussion among educators across the nation regarding how to 
close the achievement gap for boys in the area of language arts. Hands-on, art-based literacy methods 
promise to be a good match for boys and align well with research on brain differences between girls and 
boys (Gurian, 2005). 


Anecdotally, all students enthusiastically participated in these highly structured, comprehensive art-and- 
literature-based literacy models. Their level of engagement and motivation to complete each book 
project, particularly among those students who are not generally engaged by traditional verbal methods 
of instruction, was outstanding. 


Students also developed a new appreciation for the use of picture books as mentor texts, both for their 
artwork and their writing. Students came to see themselves as published artists and writers, and thus 
naturally sought out the expertise of professionally published authors and illustrators. All students 
showed a great deal of pride in the published picture books they artfully created and that motivated 
them to want to read their own and each other’s books. This fostered an enthusiasm for reading, 
especially among those students for whom reading does not come easily. Seeing the success and 
enthusiasm of their sons and daughters, parents were likewise very enthusiastic. 
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Unanticipated Challenges 


We encountered several unanticipated challenges that affected the impact of the treatment. 


Because Manchester School District had been identified as a “District in Need of Improvement” 
just prior to the beginning of our work in the schools, the District adopted several new initiatives 
that September including piloting a new math program, which created additional pressure on 
teachers. We were asked to delay our work in the classroom from September to January, thus 
reducing teachers and students exposure to the treatment during the first year. When we did 
arrive in the schools midyear, teachers expressed concern regarding how they were going to fit 
PW into their already packed school day. Thus teachers weren’t able to fully embrace and 
implement the treatment in the way we had hoped. Rather than facilitating Artists/Writers 
Workshop (and the PW/IM model) 3 times a week for the entire school year as we had initially 
planned, a vast majority of teachers facilitated the process twice a week for 2-3 months 
beginning in the winter/spring term the first year. During subsequent years, implementation of 
the treatment began in the fall and ran into the spring, with most teachers facilitating the 
process only 2x a week. Rather than embracing the treatment as an ongoing process, tended to 
view PW/IM in terms of individual book projects to be undertaken and completed with breaks in 
between. The majority of treatment teachers completed a book project with their students in 
the fall and another in the spring. We had originally anticipated that students would complete 3- 
4 book projects a year. The reading portion of the treatment (which entailed publishing 
students’ books promptly and maintaining a classroom library of students’ published work, with 
time set aside for them to read and discuss each other’s books), was never fully implemented 
due to time constraints. This may have resulted in the mixed findings on the district and state 
reading assessments. 


We also experienced tremendous turnover in administration. From the time of our early 
meetings with the district, when we first received a commitment to move forward with the 
grant proposal, to the end of the funded research period, Manchester School District had three 
superintendents. During the four-year funded period, within our three treatment schools, there 
were a total of six principals (three principals in one of the treatment schools, two principals in 
another, and one principal in the third). Incoming superintendents and principals were not as 
invested in supporting implementation of the model as those original administrators who first 
made the commitment to embrace this alternative approach to literacy learning. 


Additionally, we experienced tremendous turnover in teaching staff. Significant changes to the 
NH state policy regarding teachers’ retirement benefits resulted in an enormous turnover in 
teaching staff midway through our study. During the course of the study, we witnessed 
approximately a 50% turnover in teaching staff within our treatment schools. Despite efforts to 
train new teachers as they arrived at each treatment site, fidelity to the model was greatly 
impaired by a combination of teacher turnover and lack of administrative support in two schools. 
Interestingly, our highest performing treatment school in both reading and writing ended up 
being the one school with the highest percentage of students on F&R but which had consistent 
administrative support and less teacher turnover. This same “high fidelity school” demonstrated 
impressive gains for boys as well as Title | and economically disadvantaged students. 

36 


Due to the overall promising findings, particularly for those considered to be at-risk, and given the above 
unanticipated challenges, we recommend that a similar study be repeated to achieve a more accurate 
measure of outcomes under more typical circumstances. While there will always be limitations when 
conducting research in educational settings, especially over multiple years, the timing of this particular 
study relative to unfolding events in the District and the State proved to be most challenging. 
Designation as a “District in Need of Improvement” just prior to the commencement of our study, 
triggering several unanticipated events (new pressures on teachers’ time resulting in the postponed 
entry into the schools, the lack of time for teachers to fully embrace the treatment, the atypical amount 
of administrative turnover, and 50% turnover in trained teachers in our treatment schools midway 
through our study due to changes to NH teacher retirement benefits) made it difficult to evaluate the 
potential of this approach under more typical circumstances. 


Reflections 


Reflecting on these findings, despite the above-mentioned challenges, we learned about the universal 
appeal of Picturing Writing and Image-Making among a wide range of learners and its effectiveness in 
teaching writing to a wide range of learners, including recent immigrant and refugee students and other 
traditionally low-performing populations. We observed students benefit from the opportunity to 
participate in the Picturing Writing/Image-Making processes regularly throughout the school year as 
well as across consecutive grade levels. Likewise, we observed that teachers’ facilitation skills improve 
over time and that impacted student performance. While we were encouraged to see previously low- 
performing students make great strides, we also observed top students being challenged to stretch in 
new ways. We learned about the critical nature of administrative support on both the district and school 
building level. We strongly recommend securing administrative support as well as teacher buy-in before 
commencing a schoolwide adoption. These are key to being able to fully realize the potential of this 
innovative approach to literacy learning. 


In a time of increasing diversity within our schools and more and more state and district mandates, we 
witnessed how seamlessly the Picturing Writing/Image-Making model can be integrated across the 
curriculum. This offers great promise in allowing teachers to address national and state standards and 
district-wide curriculum while meeting the needs of their diverse community of learners. Given 
longstanding achievement gaps that exist across the nation, particularly in the acquisition of essential 
literacy skills, we hope that educators, administrators, and policy-makers will come to recognize the 
power of studying and creating visual as well as written texts to provide an engaging and effective 
alternative pathway into literacy learning for a wide range of learners. 
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Instruments 
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Text Only Scoring Instrument 


Lead sentences 
Sense of setting (time of day, weather, season, place) 
Hook 


Character traits or development 
Describes what character is doing 
Describes (shows or tells) how character is feeling 


Story development/Plot 

problem or anticipated event/drama or suspense 
solution or sense of relief 

resolution or sense of conclusion 

sense of sequence/cohesiveness 


| | 


Descriptive language 

strong adjectives 

strong verbs and adverbs 

sensory component 

inclusion of detail (including similes, personification...) 
transitional phrases 


a 


Overall quality 

Do words paint a picture in reader’s mind? 
Do words establish mood? 

Does the piece have voice? 


6. Overall use and quality of written information to tell the story 
Additional Characteristics: 


1= none, 2 = minimal, traces, 3= abbreviated 
4 = partially developed, 5 = fully developed, 6 = extraordinary 


This scoring instrument was used to evaluate the quality of students’ story writing. 
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Use of Visual Information to Communicate Ideas Scoring 
Instrument 


1. Color 
Color accurately represents setting elements 
Color used to enhance mood 


2. Texture 
Texture used to represent story elements 
Texture used to enhance mood 


3. Shape 
Shape/contour used to represent setting 
Shape attempts to accurately represent key objects 


4. Detail 
Details used to enhance setting 
Details used to depict character 


5. Composition 
Foreground/background provide information about setting 
Placement/size creates sense of depth 
Placement/size used to enhance sense of importance 
Picture composition conveys meaning of text 


6. Sequence 
Sequence reflects changes in time of day/weather, etc. 
Changes in perspective enhance the story 
Picture sequence tells the story 

7. Overall use and quality of visual information to tell the story 


1= none, 2 = minimal, traces, 3= abbreviated 
4 = partially developed, 5 = fully developed, 6 = extraordinary 


This visual instrument was used to evaluate students’ use of visual 
elements to make meaning and communicate their ideas. 
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